1. Technical Field
The present invention relates to video games in general, and in particular to interactive video games. Still more particularly, the present invention relates to a method and apparatus for providing real-time machine learning to computer-controlled agents used in interactive video games.
2. Description of Related Art
As the video game market continues to expand, video games have become a facet of many people's lives. One of the most compelling, yet least exploited, technologies that can be applied to video games (or other interactive simulation applications) is machine learning.
In the field of video games, the term non-player character refers to an autonomous computer-controlled agent in a video game. The behavior of an autonomous computer-controlled agent in current video games is often repetitive and predictable. The scripts in most video games simply cannot learn or adapt to change computer-controlled agents, and as a result, opponents will always make the same moves and the video game quickly becomes boring.
Machine learning can potentially keep video games interesting by allowing computer-controlled agents to change and adapt. However, a major problem with machine learning in video games is that if the behaviors of computer-controlled agents are allowed to change without any type of control, the game content can become unpredictable. For example, computer-controlled agents may learn idiosyncratic behaviors or even not learn at all, making the gaming experience of a video game player unsatisfying. One way to avoid such a problem is to train computer-controlled agents to perform complex behaviors offline, and then upload the results into the final version of the video game intended for public consumption. However, although the video game will become more interesting with offline machine training, computer-controlled agents still cannot adapt and change in response to the tactics of a skillful video game player.
Consequently, it is desirable to provide a method for providing online machine learning to computer-controlled agents used in video games such that the computer-controlled agents can adapt and change in real-time during game play.